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IMPROVING SCIENCE INSTRUCTION IN 
SOUTHWESTERN ILUINOIS AND METRO EAST ST. LOUIS: 
STUDENTS LEARNING SCIENCE THROUGH A SUSTAINED 
NETWORK OF TEACHERS 


This article describes the specific methods of a regional partner¬ 
ship that has lasted more than twenty-five years. Southern Illinois Uni¬ 
versity Edwardsville has partnered with public and private schools in the 
southwestern portion of Illinois, and in metro St. Louis, in the Hands-On 
Science project, which provides instruction development for chemistry’, bi¬ 
ology’, earth science, and physics teachers in grades 6-12. The project 
has been modified over the years, and this article explains both modi¬ 
fications and causes, in specific terms such as the effects of funding on 
classroom equipment and supplies as well as, more importantly, on the 
turnover among teacher participants. The Hands-On Science project em¬ 
phasizes classroom assessment techniques, collaboration, and more ac¬ 
tive learning. Quick Course Diagnosis and Group Instructional Feedback 
Technique (or Small Group Instructional Diagnosis) are two of the specif¬ 
ic methods of assessment used in this successful, long-lived ITQ project — 
which originally began as an Eisenhower project. 


Need for Content Based Professional Development Activities 

A key goal of the Illinois Board of Higher Education (IBHE) stra¬ 
tegic plan, Illinois Public Agenda for College and Career Success, ad¬ 
dresses a disturbing and growing ‘“prosperity gap’ that relates to large 
and widening disparities in educational attainment” and advances a crit¬ 
ical role for higher education partners to help create the conditions for 
prosperity (IBHE, 2008, p. 1). In the 21st century, this must surely in¬ 
clude extending high quality Science, Technology, Engineering, and Math 
(STEM) education to every region of the state. Southern Illinois Univer¬ 
sity Edwardsville (SIUE), the university partner in this case study, is in the 
southwestern region of the state, near the Mississippi River and metro St. 
Louis. The rural, urban, and suburban districts of the region have differ¬ 
ing needs for professional development and teacher learning overall, but 
they all require a university partner to prepare and then continuously up¬ 
date the teachers in the region in the dynamic STEM disciplines. This ar¬ 
ticle for the special issue of Planning and Changing discusses how SIUE 
has developed as a regional partner for schools and districts for more than 
25 years, originally funded as an Eisenhower program, and now funded as 
an Improving Teacher Quality (ITQ) state grant program. 

We have found that public and private school teachers crave pro¬ 
fessional development and learning opportunities. It is widely known that 
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we learn about things by teaching about them. Only when they face their 
own students do our middle, junior high, and high school teachers viv¬ 
idly realize the importance of what they encountered as undergraduate 
science majors. By then, they have forgotten or incorrectly remembered 
many core concepts. If they stumble in their teaching, they propagate un¬ 
truths and inadequate foundations in science, and they also communicate 
confusion and disinterest. Our project, Hands-On Science, addresses these 
conditions head-on. As the teachers learn science content and processes 
for lab and field work from us, we learn about the conditions in schools 
that allow us as partners to contribute to college readiness on the part of 
incoming SIUE students through teachers’ professional learning. 

In his book, Designing Powerful Professional Development for 
Teachers and Principals, Dennis Sparks (2002) describes how professional 
development of teachers and administrative leadership play central roles in 
connecting the quality of teaching to the improvement of schools. Profes¬ 
sional development became an explicit policy with the No Child Left Be¬ 
hind Act, and Illinois policy mirrors this trend (Desimone, Smith, & Phil¬ 
lips, 2007). Our intent is to work with regional grades 6-12 science teachers 
to help them improve their science knowledge, gain insight into applications 
of science, and become aware of available high quality science education re¬ 
sources and regional business partners. These tools and knowledge should 
assist teachers in preparing their students to meet the standards expected in 
middle, junior high, and high school science courses. It is anticipated that 
these activities will eventually lead to students who are better prepared for 
high school and college science courses, particularly students from histori¬ 
cally under-represented and under-served groups. In a study that used a na¬ 
tional probability sample of 1027 teachers to determine the effects of differ¬ 
ent types of professional development on teachers’ learning: 

...results indicate that sustained and intensive professional de¬ 
velopment is more likely to have an impact.... Our results also 
indicate that professional development that focuses on academic 
subject matter (content), gives teachers opportunities for ‘hands- 
on work’ (active learning), and is integrated into the daily life of 
schools (coherence), is more likely to produce enhanced knowl¬ 
edge and skills (Garet, Porter, Desimone, Birman, & Yoon, 2001, 
p. 935). 

Garet and his colleagues extended these insights to name five elements of 
effective professional development: (a) duration (i.e., “sustained” and “in¬ 
tensive); (b) collective participation; (c) content focus; (d) active learning; 
and (e) coherence with local policies (Desimone, 2009; Garet et al., 2001). 
This empirically grounded model of exemplary professional development 
guides the design of partnership structures and influences the implementa¬ 
tion and evaluation of all ITQ grants. In recent years, under the auspices of 
ITQ, this research has been used by the partners in the Hands-On Science 
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project. The Departments of Biology, Chemistry, Physics, and Geography, 
and the Environmental Sciences Program at SIUE have developed internal 
partnerships with the SIUE College of Arts and Sciences, the SIUE School 
of Education, and external partnerships with over 70 partner middle, ju¬ 
nior high, and high schools (i.e., grades 6-12) and school districts to im¬ 
prove science education in the southwest region. 

Elistory of the Project 

This partnership began in 1985 as a professional development 
project for high school chemistry teachers with funding from the feder¬ 
al ESEA Title II Dwight D. Eise nh ower Professional Development Pro¬ 
gram. A consortium of partners from SIUE, Southern Illinois University 
Carbondale (SIUC), and Eastern Illinois University (EIU) provided chem¬ 
istry workshops for high school teachers from districts throughout south¬ 
ern Illinois. SIUE project directors were Emil Jason and Sadegh Khazaeli. 
Over time, SIUC and EIU partners no longer participated, leaving SIUE 
as the only university partner. Chemistry workshops were offered as once- 
a-week evening classes in spring semesters, and as all-day classroom and 
laboratory experiences for a week in summer semesters. Sadegh Khazaeli 
continued as project director after Emil Jason’s retirement. 

In 2004, the partnership continued in the form of the Hands-On 
Chemistry project, funded by the No Child Left Behind (NCLB) Title II 
program. Eric Voss joined as project co-director of the Hands-On Science 
project, to also include middle and junior high school science teachers in a 
two-week summer professional development workshop. Starting in 2007, 
high school physics workshops in summer and fall were added with assis¬ 
tance from Abdullatif Hamad, and then in 2010, fall high school biology 
workshops were added with Dennis Kitz as the content expert. Currently, 
ten SIUE faculty Contribute to the content expertise for this project. 

Project Description 

The goal of this project is to provide hands-on professional devel¬ 
opment for high school chemistry, physics, and biology, and middle and 
junior high school science teachers with an emphasis on science subject 
matter related to the Illinois State Board of Education’s Illinois Learning 
Standards (ISBE, 2010a) and Illinois Professional Education Standards 
(ISBE, 2010b). Project directors, an external evaluator, other SIUE fac¬ 
ulty, and administrative leaders work with regional teachers to help them 
improve science content knowledge, gain insight into applications of sci¬ 
ence, and increase awareness of available high quality science education 
resources and innovative student learning assessment techniques. 

Working in collaboration with regional partner educators, we have 
devised the Hands-On program, with these key goals: (a) development of 
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participants’ understanding of the basic principles of science to a sophisti¬ 
cated level; (b) development of participants’ laboratory and/or field skills; 
(c) sharing of resources, ideas, and experience among the participant edu¬ 
cators; (d) establishment of a sustained teacher and school administrator 
network among participants; and (e) a considered, flexible response to ed¬ 
ucators’ needs for determining the future direction of the project. 

We have provided area science teachers with the opportunity to 
develop a deeper understanding of science content, to integrate this under¬ 
standing into their 6-12 curricula, and to implement simple yet effective 
tools to assess student learning in science. The overarching goal of all of 
these activities is to improve the learning of science content and applica¬ 
tion by Illinois students in the region we serve. 

Project partners each year include about 150 teachers representing 
more than 70 school districts, and both public and private schools. We es¬ 
timate that 20,000 students are impacted by program activities each year. 
Table 1 summarizes partners in the project over the past four years as an 
ITQ project. 

Table 1 


Partners in the Project for the Past Four Years 


Year 

Workshop 

Total number 
of teachers 

Teachers from 
public schools 

Teachers from 
private schools 

2007 

Middle school science 

42 

33 

9 


High school chemistry 

68 

66 

2 


High school physics 

27 

27 

0 

2008 

Middle school science 

50 

39 

11 


High school chemistry 

66 

62 

4 


High school physics 

35 

35 

0 

2009 

Middle school science 

51 

38 

13 


High school chemistry 

66 

64 

2 


High school physics 

27 

27 

0 

2010 

Middle school science 

43 

34 

9 


High school chemistry 

57 

52 

5 


High school physics 

27 

27 

0 


High school biology 

16 

13 

3 


Special Features 

SIUE programs have been successful in encouraging networking 
and sharing of resources among a core group of teachers in the area, some 
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who have known each other for many years. For example, one Granite 
City Fligh School chemistry teacher, Amy Fleath, was a first year teacher 
when the partnership began in 1985, and continues to be an active collab¬ 
orator 25 years later. Current activities reach many educators, including 
those serving financially disadvantaged and/or mainly minority districts. 
Ongoing collaborations have produced highly qualified and effective sci¬ 
ence teachers who serve in urban, suburban, and rural districts. 

School educators and administrators are responsible for implement¬ 
ing curricular materials, experiments, and demonstrations in their science 
courses. As part of the partnerships’ formal agreements, each school or dis¬ 
trict is required to commit to providing lab space and offering hands-on sci¬ 
ence to students, although the quality of these accommodations varies wide¬ 
ly. This policy ensures some understanding by administrators and a baseline 
of commitment to support teachers in implementing what they have learned. 
The partnership includes formal planning, oversight, direction-setting, and 
decision-making by a range of partners. Throughout the year, a Program 
Board meets on a regular basis (approximately every other month). The Pro¬ 
gram Board consists of the project director (Sadegh Khazaeli), co-project 
directors and science faculty (Eric Voss, Dennis Kitz, Abdullatif Flamad, 
and Cathryn Springer), a representative from the SIUE College of Arts and 
Sciences (William Retzlaff), a representative from the SIUE School of Edu¬ 
cation (Randy Smith), the external evaluator (Douglas Eder), ten “liaison” 
teachers, and two science department heads from partner schools, both ac¬ 
tive Hands-On teachers as well. The Program Board acts as a “sounding 
board” for communication in both directions between SIUE partners and 6- 
12 partners. Each “liaison” in turn communicates with five to six districts, 
follows up on their evaluation processes and represents their views at all 
meetings. This is believed to be an effective method for continuous feed¬ 
back and serves as one tier of a network that provides material, personal, and 
professional support in the schools and districts. 

Nine high need school districts are targeted as principal partners of 
this program: East St. Louis #189, East Alton #13, East Alton-Wood River 
High School #14, Belleville #118, Cahokia #187, Madison #12, Venice #3, 
Centralia High School #200, and Centralia #135. Local data obtained from 
four other school districts identify them as high need districts: Belleville 
East, Edwardsville, Roxana, Granite City, and SIUE East St. Louis Charter 
School. Other public schools that have participated in previous programs are: 
Highland, Okawville, Valmeyer, Greenville, Mt. Olive, Triad, Mascoutah, 
Gillespie, Belleville West, Carlinville, O’Fallon, Staunton, Civic Memorial, 
Waterloo, Lebanon, Livingston, Collinsville, New Athens, Freeburg, Diver- 
non, BCCU #2, Jersey, Mulberry Grove, Bond County, Coolidge, Damains- 
ville, Grantfork, High Mount, Highland Elementary, Hillsboro, Irvington, 
Liberty, Lincoln, Pocahantas, Pontiac, Ramsey, Smithton, Belleville West 
Junior High, Bunker Hill, Fulton, Garant, Virden, Nashville, Mary Schae¬ 
fer, Joseph Arthur, Grigsby, Fox, Mel Carnahan, North Green, Red Bud, and 
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Southwestern. The private schools that have participated in this program 
are: Trinity Lutheran, Holy Trinity, St. Anthony, Governor French Academy, 
Holy Rosary, Holy Childhood, Christian Academy, St. Ambrose, St. Clare, 
St. James, St. John the Baptist, St. Rose, Mater Dei, Christ Light of the Na¬ 
tion, Gibault Catholic, St. Elizabeth, and St. Michael. 

Challenges 

The SIUE network of partnerships has several strong features, two 
of which are longevity and network density. The two features are naturally 
related, and several of our challenges are the result of successes. First, the 
number of participants is growing each year. A very real challenge is find¬ 
ing appropriate laboratory space on campus to accommodate all partici¬ 
pants. Hands-on science requires that teachers have genuine lab and field 
experiences themselves so that they can witness their value and learn how 
to provide comparable experiences to students. Because of safety consid¬ 
erations, the number of teachers in a given laboratory room should not ex¬ 
ceed 20. SIUE provides several laboratory spaces for each workshop, even 
though these rooms are at a premium, as a sign of SIUE’s commitment to 
the network of partnerships. We also split groups larger than 20 into sub¬ 
groups doing different activities in different laboratories at the same time. 
This kind of coordination requires planning and university support. For 
these sessions, we have two faculty and two teaching assistants supervis¬ 
ing the subgroups. If growth continues, we will offer more workshops and 
find ways to address the costs. On the average the cost for each participant 
is about $2,000. Stipends, enrollment fees, demonstration kits, and com¬ 
modities for use in the schools amount to about 2/3 of the total cost. 

Second, there is a great diversity in the experiences of teachers 
attending project sessions. Some teachers have just started their careers, 
while others are veteran teachers with more than 20 years of experience, 
and there is great variation in the content knowledge they bring with them. 
The ongoing challenge here is whether to focus on basics or to focus on ad¬ 
vanced content. Unlike most ITQ projects, we have so many school part¬ 
ners that we must offer content without having a specified curriculum or 
school improvement initiative. So far, we have tried to maintain a balance 
between basic and advanced material guided by science standards, and in 
general, school partners have liked our approach. We also plan workshops 
considering new developments in science of which teachers may be un¬ 
aware. For example, advanced techniques such as gas chromatography 
and mass spectrometry experiments have been included in each summer 
session so that teachers are exposed to techniques currently used at univer¬ 
sities and in industry. Although these techniques are not part of the basic 
curriculum and current standards for middle, junior high, and high school 
science, they are a part of modem science, and they provide examples of 
how science is used to improve the lives of people in society. 
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Third, teachers have suggested that we offer similar science con¬ 
tent-based programs in other disciplines, but it is challenging to form the 
internal partnerships at SIUE that would make this possible. A related chal¬ 
lenge is extending all the sciences to middle, junior high, and high school 
teachers. The first population we served was chemistry teachers, one group 
covered by the current project. The middle and junior high school science 
teacher program has been in place for six years, and continues to attract a 
diverse group of teachers. Starting in 2007, our project also includes work¬ 
shops for high school physics teachers. We expanded project activities to 
include earth science and environmental science components in our middle 
school science and high school chemistry workshops. We started similar of¬ 
ferings for high school biology teachers in fall 2010. Because of identified 
school needs, we would like to add mathematics workshops in the future. 

Fourth, the IB HE has encouraged all NCLB projects to include as¬ 
sessment and evaluation feedback loops among all partners involved. To 
address this challenge, we now have meetings throughout the year of the 
Program Board described above. With the growth we have experienced 
in recent years, we expanded the Program Board in 2010 to include two 
high school physics teachers, three high school chemistry teachers, and 
five middle and junior high school science teachers. We also invited four 
school administrators to one of our Project Board meetings in hopes of 
securing commitments from two administrators to become Project Board 
members. We currently have the commitment of two science department 
chairs, one from a suburban community and another from a high school 
that serves several rural communities. Taken together, these science ad¬ 
ministrators serve a southern Illinois population of approximately 60,000 
people. We aspire to adding two building level administrators, principals 
or assistant principals from project schools, to the board in the future. In¬ 
ternal partners at SIUE, including representatives of both Arts and Scienc¬ 
es and Education Deans, serve on the board as well, providing means to 
secure support for program changes based on evaluation results and input 
from school partners. 

Fifth, we have used SIUE School of Education contacts and for¬ 
mal agreements with area school districts and have expanded interactions 
between project partners and area school administrators. This addition¬ 
al participation by administrators is crucial to bringing about significant 
systemic change and better access to schools for student learning assess¬ 
ments. Leadership matters at all partnership intersections, between and 
among university internal partners and school partners. Originally, our 
partnerships were single-tier professor-to-teacher structures that have 
been evolving into multi-tier arrangements with leadership engagement 
and a shared awareness of what a sustainable partnership requires (Baker, 
2011; Gardner, 2011a). Leadership stability and commitment is a source of 
persuasion and buy-in for improving science education that translates into 
the five professional development features in the IBHE policy (Desimone, 
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2009; Garet et al., 2001). SIUE science faculty are the content experts 
who design summer workshops and school year follow-up programming 
to ensure sufficient active learning, duration, and collective participation 
to make the project viable and sustainable. However, the school leaders 
must include these features and ensure that this science professional devel¬ 
opment initiative is coherent with other school improvement policies and 
does not represent a source of fragmentation in the busy lives of teachers. 
In schools where the goals of the project can be supported by collective 
participation in school-based teams, leaders are critical to the functioning 
and sustainability of these teams. 

Finally, some of the schools do not have even basic pieces of equip¬ 
ment and commodities for implementing program activities. As part of for¬ 
mal partnership agreements, schools must commit to do what they can to 
provide basic infrastructure for hands-on science, but we are sensitive to 
the different contexts in our large regional network. Without the partner¬ 
ship, frankly, SIUE faculty in the sciences might be unaware of school con¬ 
ditions that affect college readiness. Additional funds provided by IBHE 
in 2009 enabled us to purchase equipment and materials for schools in the 
partnership. Other more expensive pieces of equipment purchased using 
project funds are stored at SIUE and are available for loan to teacher part¬ 
ners. The library of equipment benefits all the schools and increases the 
strength of the network because teachers confer about the equipment, the 
logistics of sharing it and ideas about its uses in hands-on science. 

In the process of meeting these and other challenges, the partner¬ 
ship reveals its strengths and exposes places where partners could be more 
responsive to each other. The Program Board is a critical structure for imple¬ 
mentation, for evaluation and evaluation capacity building, and a forum for 
the development of evaluation instruments and application of evaluation re¬ 
sults to planning and improvement (Haeffele, Hood, & Feldmann, 2011). As 
a longstanding science education regional partnership, meeting these chal¬ 
lenges has helped us to realize that we do have a strong regional network that 
could not be easily or quickly developed. We can now continue to strengthen 
and use our network. We continue with some of the lessons learned about 
supporting science education through school-university partnership. 

Lessons Learned 

Demand is high for professional development among middle, ju¬ 
nior high, and high school science teachers. We have tried to accommodate 
the growing number of participants within the constraints of classroom 
and laboratory size, available staff, and monetary support. We are proud to 
have never turned away any interested participant. This was possible since 
SIUE supplied partial salaries for program personnel. Both partners have 
to commit, and the commitment of staff, space, and time for collaboration 
all serve to support partnership functions. In addition, our partnership has 
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taught us several lessons that might help in forming new partnerships as 
well as in improving longstanding ones like ours. 

First, partners must be responsive. For example, we learned that 
among high school chemistry teachers and middle and junior high school 
science teachers, there is very high interest in core academic content relat¬ 
ed to environmental science li nk ed to state standards. Based on feedback 
from earlier years of the project, we initially added one day of environ¬ 
mental science to each workshop. Teachers wanted even more environ¬ 
mental science, so we recruited Judy Zhang from the SIUE Department 
of Chemistry and Environmental Science Program to collaborate on the 
project. As a result, we were able to provide two days of environmental 
science activities in 2009 and 2010 for high school chemistry teachers and 
middle and junior high school science teachers. 

Second, partners continue to demand a focus on core science sub¬ 
ject content rather than pedagogy or educational methods. Our expertise 
is as content providers, and this is the real strength of this program. But 
content is never divorced from the ways in which science is done. Ac¬ 
tive learning, a core feature of exemplary professional development, is 
what we do as scientists in our own lab and field work. By inviting sci¬ 
ence teachers to do authentic lab and field work, we model the overlap 
between content and pedagogy. This is an exemplary professional devel¬ 
opment practice as pedagogical processes taught in isolation do not im¬ 
prove teaching effectiveness (Desimone, 2009). Assessment is also a criti¬ 
cal feature of good instruction and since Douglas Eder, (project external 
evaluator) joined the project, we have included more explicit pedagogical 
and assessment techniques in project activities. In particular, Doug has de¬ 
veloped assessment activities that are used in the schools, and we discuss 
these below. As teachers increase the assessments of learning and share 
assessments developed through the partnership, teachers not only learn 
about assessment, they develop evidentiary sources for the partnership to 
use in its ongoing cycles of evaluation and project improvement. 

Third, many partner teachers have recently changed teaching as¬ 
signments to new disciplines, a fact of life in schools stressed under eco¬ 
nomic and other pressures that create instability. We must be sensitive to 
the needs of those who may be teaching a subject for the first time with lit¬ 
tle or no prior formal training in that subject. In other cases, rural schools 
may have just one science teacher who will not have comparable levels 
of content knowledge or comfort in all the different scientific disciplines. 
This is an ongoing challenge to support teacher partners as they must re¬ 
tool and perhaps teach out of their comfort zones. It is important to have 
activities geared to both new teachers and seasoned veterans who may be 
teaching solo or out of their field of preparation, with or without recent up¬ 
dating in content knowledge, or experience with the content pedagogy of 
the field. This mixture of experience levels and content or content-peda¬ 
gogical experiences makes for interesting interactions. When asked, part- 
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ners prefer to keep the group heterogeneous rather than separating into 
groups based on experience or other factors. This matches the five-part 
model that encourages collective participation and builds a support net¬ 
work wherein teachers can benefit from the strengths of others and mutual 
support in situations where there are no science colleagues. 

Fourth, we have found that an emphasis on basic and advanced 
math skills is essential to successful applications of science, but not all re¬ 
gional science teachers are comfortable with mathematics. In our work¬ 
shops and supports, we help participants gain confidence in their math 
skills, which will certainly improve their effectiveness in teaching how to 
solve scientific problems. In addition to math as a basic prerequisite for 
science, there is great need for content on science basics, including logis¬ 
tics. We have included training in laboratory safety and stockroom man¬ 
agement for high school chemistry teachers, as one example. Also, we 
have chosen to go beyond the basics in some selected advanced topics, 
and introduce topics related to the management of specialized laboratory 
experiments. Recently, organic chemistry synthesis was included as just 
such an advanced topic. 

Fifth, we have learned the value of structured discussion time dur¬ 
ing workshops. In 2007 we added more time for roundtable discussions 
that were popular, but this was not enough. Since then, we have included 
more time at the conclusion of laboratory and field activities to discuss 
data analysis, interpretation, comparisons, purposes, improvements, miss¬ 
ing pieces, adaptations, and other topics. This provides an opportunity for 
the development of critical thinking skills. At the end of each workshop, 
participants are invited to share favorite resources (e.g., experiments, web 
pages, or teacher made print resources), and this provides another oppor¬ 
tunity for discussion and network-building. 

In addition, some partners suggested inclusion of the Illinois Sci¬ 
ence Assessment Framework (ISBE, 2010c) that was implemented as part 
of the Illinois Standards Achievement Test (ISAT) beginning in spring 
2006. We then incoiporated the framework for grades 4 and 7 and the 
framework for grades 9-12 (ISBE, 2010c) from the Illinois State Board of 
Education into all program activities. We encourage teachers involved in 
the program to be aware of new standards that are in preparation, particu¬ 
larly the common core standards. Looking ahead, international standards 
and the Programme for International Student Assessment (PISA) will in¬ 
creasingly affect science educators in this standards-based environment 
(Organization for Economic Cooperation and Development, 2009). Teach¬ 
ers and students must think ahead to the ways in which STEM education 
will continue to shape standards and expectations. 

Also, we learned that science field trips by teachers not only bene¬ 
fit them in improving their knowledge, but also the trips give them an edge 
in the classroom in explaining complicated scientific techniques. Recent 
field trips include the A nh euser-Busch brewery, the Sigma-Aldrich Chem- 
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ical Company, Argonne National Laboratory, the LEED Platinum certi¬ 
fied Alberici Corporate Eleadquarters, and TekLab, an environmental test¬ 
ing laboratory. More could be done in this area as we explore the role of 
science in regional development, including the universal need for science 
background for 21st century employment and how it potentially affects 
our region. Not all 21st century STEM employment will require advanced 
education. The region needs lab and pharmacy technicians, physical ther¬ 
apy aides, and other jobs that contribute to regional economic health and 
opportunities for students, college bound or not. 

Finally, the outside evaluator for this grant, Douglas Eder, makes 
the observation that strict memorization of facts is not how practicing sci¬ 
entists “do” science. Nevertheless, a focus on facts and content has become 
a default approach in science teaching, one that is remarkably resistant to 
change. A lesson learned—or at least reinforced—is that one cannot skip 
steps during progress toward a long-term goal. Until teachers are comfort¬ 
able and confident with what they want to learn, which is content-oriented, 
they are resistant to moving toward what we want them to learn. Even though 
the big payoffs for student learning may exist outside the strict boundaries 
of science content, the teachers won’t reach for these payoffs until they un¬ 
derstand the prerequisite material. Stated in assessment terms, the best aca¬ 
demic predictor of what students will learn is what they already know. If the 
teachers aren’t comfortable with content, they have great difficulty in going 
to the next level and requiring a greater cognitive demand of their students. 

Evaluating Content-Based Professional Development 
in a Regional Network 

The IBHEITQ projects evaluate and use evidence about each proj¬ 
ect for ongoing improvement at the project level and for evaluation capac¬ 
ity within and across projects (Haeffele et al., 2011). Hands-On Science at 
SIUE has a theory of change rooted in the value of content expertise and 
discipline-specific approaches that include investigative laboratory and field 
work. Critical to the partnership is the content expertise offered by the uni¬ 
versity, which now includes multiple science departments, as well as the 
value of the network in making the project viable and sustainable (Gardner, 
2011b). IBHE further requires that projects create logic models to make the¬ 
ories of change explicit for a variety of audiences, as an aid to planning and 
implementation, and as a guide to evaluation (Frechtling, 2007). In this sec¬ 
tion, we offer the SIUE logic model and some comments on evaluation. 

Logic Modeling 

In 2008, we began to use logic modeling with the tools, examples, 
and training provided by the Illinois Board of Higher Education (IBHE; 
see Figure 1). ITQ projects are required to evaluate teacher learning, im- 
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plementation, and student learning using the logic model as a representa¬ 
tion of intended outcomes and a set of pathways for evaluation. Using our 
logic model, we started to explore new ways of gathering evidence about 
what the project has accomplished. Logic models link inputs or resources, 
activities, outputs, and outcomes in a linear fashion to provide a road map 
for evaluation (Frechtling, 2007). Our project’s theory of change empha¬ 
sizes professional development in content knowledge and science practice 
and has done so for 25 years. What logic modeling suggested to us was 
the role of the regional network as part of our theory of change for both 
school partners and the university that accrues the benefit of science readi¬ 
ness in first year students. We include some of these emerging measures 
and means of observing project outcomes we are developing. 


INPUTS ACTIVITIES OUTPUTS OUTCOMES 



Feedback 


Figure 1. Program theory/logic model—Part 1: teachers and students. 

Evaluation: Observations and Measures 

The Hands-On Science logic model (Figure 1) is a road map for 
understanding our approach to designing, implementing, and evaluating. 
The evaluation methods employed within this project were aimed chief¬ 
ly at the outcome level, and most often at examining student learning, al¬ 
though we looked also at teacher learning and implementation using sev¬ 
eral methods that we share below. We also used a survey to yield findings 
about the Hands-On Science regional network and communities of prac¬ 
tice. Because of our interest in student learning and helping teachers use 
classroom assessment to inform instruction, many assessments were drawn 
from, or were variations of, classroom assessment techniques (CATs), and 
we piloted the use of CATs by a small group of teachers (Angelo & Cross, 
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1993). Four additional instruments we used included: an on-line survey; 
embedded questions from a common exam; the Quick Course Diagnosis 
(QCD) (Lansing Community College, 2010); and evaluations of teacher 
learning. We summarize these topics and methods below. 

In addition to science content, teachers received instruction on a 
dozen classroom assessment techniques (CATs), such as the one-minute 
paper, directed paraphrase, memory matrix, rapid organizer, and double 
entry journal. These and other CATs were selected because they required 
little teacher preparation, yet they did require student reflection and could 
make learning visible as it happened, expanding a teacher’s assessment 
repertoire. As a result, a dozen teachers out of the 150 or so involved in 
this project in 2010 voluntarily reported CATs results. This is a small num¬ 
ber, but this interest in CATs was suggestive and served to pilot several ap¬ 
proaches for future use and to identify areas for further investigation. Is¬ 
sues raised by this small sample revealed that students: (a) did not focus 
reliably on what their teachers said was important; (b) too often omitted 
important material that was essential for understanding and frequently rat¬ 
ed as important material that was largely irrelevant; and (c) desired more 
active learning experiences and fewer pure lectures. One implication of 
these findings from this small sample is that the project could use CATs 
more systematically and help teachers to make reflective use of student 
communications. Teachers can have the classroom feedback they need 
to alter instruction, clarify goals for themselves and their students, and 
to communicate about learning goals, including learning standards. Stu¬ 
dents need to be conversant in the world of standards and use them to take 
charge of their own learning (Black & Wiliam, 1998), so CATS are poten¬ 
tially useful tools for all units of analysis in our evaluation. 

CATs are emerging as useful formative assessment tools, but we 
use other methods of evaluation that are not as closely linked to instruction. 
Surveys are an important evaluation of Flands-On Science. Participating 
teachers are invited to complete 9-question, on-line surveys at the end of 
each fall semester. Response rates each year were between 58-70%, and 
we strive to improve this rate but still find results instructive. Unlike atti¬ 
tude surveys, which ask about impressions, feelings, and interpretations, 
these surveys elicit factual information about teacher behavior. For exam¬ 
ple, one question asked teachers to count the number of contacts (e.g., tele¬ 
phone, e-mail) they made with other teachers each week on project-related 
topics. The contacts could be to check facts, exchange supplies, acquire 
pedagogical assistance, borrow equipment, or something else so long as it 
was grant-related. The most important finding was the prevalence of net¬ 
working. Participating teachers were communicating with each other on 
grant-related topics at the rate of an average of four contacts per school 
day across the network. Given the isolation that teachers often profess— 
e.g., “I’m the only chemistry teacher in my school”—these contacts were 
important and would not have existed except for the project. The contacts 
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themselves facilitated the borrowing and exchanging of ideas, equipment, 
and supplies. Through the surveys, teachers also reported subject areas in 
which student learning was improving and those in which it was not. This 
information informed the principal investigators so that they could tailor 
future workshops and interventions: Would teachers benefit from more 
emphasis on hypothesis forming and testing? Or more insights on colliga- 
tive properties? Less emphasis on weights and measures? In this way, we 
used surveys to address teachers’ needs directly and to understand how the 
project was being implemented. 

The Program Board meets two to three times each semester to pro¬ 
mote evaluation and use of evaluation results. One important advance from 
this group was the construction of two student exams, one in chemistry 
and one in physics. The exams were designed to be used as stand-alone as¬ 
sessments of content learning or as questions to be embedded in other ex¬ 
ams and then retrieved for item analysis. Remarkably, the questions were 
designed for use by teaching participants in the project and by non-par¬ 
ticipants as well, in order to assess student learning in “intervention” and 
“control” situations. Pledges were received from participating teachers to 
solicit cooperation (and future participation) by non-participants. These ex¬ 
ams were administered throughout the Hands-On network in spring 2011, 
but results of this assessment were not yet available at the time of publica¬ 
tion. We believe that this method is promising for evaluation at all levels of 
analysis from the classroom to the Program Board. 

As a further investigation of classroom learning, we trained teach¬ 
ers in the Quick Course Diagnosis (QCD), a derivative of the longer and 
more complex Group Instructional Feedback Technique (GIFT), some¬ 
times called Small Group Instructional Diagnosis (SGID). The key ele¬ 
ment is for groups of four to six students to respond to two questions: (a) 
what aspect of this course helps you learn? (b) What aspect hinders your 
learning? The reason for using groups is so that no one student voice dom¬ 
inates. If each group can return only one response to each question, then 
group agreement becomes essential. For a typical class of approximately 
30 students, this means that each question produces five to seven group re¬ 
sponses. In addition, the emphasis of the questions is on the learning envi¬ 
ronment, not on the teacher’s behavior. This is a learning assessment, not 
a teaching evaluation. The major difference between the GIFT/SGID and 
the QCD is that the GIFT is done through a one-half-to-l-hour classroom 
interview process whereas the QCD is accomplished via a 10-minute 3x5 
card process. In both processes, a trusted teaching colleague conducts the 
class with the host teacher absent in order to assure student anonymity. 
The trusted colleague also transcribes and summarizes the data and pres¬ 
ents it privately to the host.. .hence the idea of a “gift.” We found consid¬ 
erable agreement observed across teachers, subjects, and schools. Students 
said that they learned better when given active learning situations, such as 
lab exercises, opportunities for group processes, repetition and practice, 
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slower talking by the teacher, and time in class for reflection. Things that 
students reported to hinder their learning included too much lecture, fast 
talking by the teacher, no time during class for practice and reflection, and 
both aural chatter and visual clutter in the classroom. 

Finally, all participating teachers completed evaluations during 
and after each workshop or follow-up session. The evaluations asked for 
feedback regarding pedagogy as well as content. Workshop facilitators 
took the feedback to heart and now more consistently model effective and 
engaging teaching practices. Moreover, the principal investigators paid 
special attention to those topics that teachers said they would find most 
useful in their own classrooms: Does chromatography of flower pigments 
sufficiently engage students to convey an important point at low expense? 
Does making ice cream get across the idea of a colligative property (i.e., 
freezing point depression), or is the exercise merely distracting? (To make 
ice cream, one doesn’t need a traditional barrel with dasher. There are in¬ 
expensive, colorful, plastic balls available with an inner compartment in 
which cream can be cooled by icy slush laced with salt in an outer com¬ 
partment. Agitation occurs when students roll the ball on the floor. To what 
extent is the thermodynamic mechanism of salt depressing the freezing 
point of water, thereby freezing the cream, made clearer and memorable 
by making ice cream?) The workshop evaluations prompted these sorts 
of questions, and the answers were used to reinforce or restructure sub¬ 
sequent grant-supported interventions. Thus, the participating teachers’ 
own knowledge base was reinforced (and sometimes recast!) in a manner 
aimed to yield subsequent improvements in student learning. 

Some Challenges 

From an external evaluator’s viewpoint, five challenges emerge. 
First, in an era of financial distress, it was discouraging to observe that 
some of the most committed teachers were laid off and could not serve 
the schools that had sent them to Flands-On Science. Teachers who are 
bright and energetic applied to become, and were accepted as, participants 
in the training afforded by this project. If coherence, duration, and col¬ 
lective participation are hallmarks of effective professional development, 
the factors that work against them are discouraging. These factors includ¬ 
ed the fragmentary nature of state support and lack of instructional pro¬ 
gram coherence in the schools (Newmann, Smith, Allensworth, & Bryk, 
2001). Looking first at the state, the IB FIE administered this program and 
supported evaluation looking to improve schools and districts, but the 
state legislature reduced funding for schools. In this way, grant funding 
can be a source of support, but it is seldom integrated into a system that 
helps schools use fiscal and human resources to their best effects. A second 
source of fragmentation comes from the districts. The lay-off decisions 
that affected Hands-On Science were unrelated to who was seeking mean- 
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ingful professional learning and who was not. Failure to align staffing de¬ 
cisions with a coherent approach to school improvement compounded our 
frustration as good science teachers leave project schools and districts, al¬ 
though recent legislation in Illinois may change teacher lay-off priorities 
as a matter of policy. 

Second, evaluation efforts are hampered by confusion about the na¬ 
ture of formative evaluation. During the course of using the GIFT/SGID, 
some school administrators learned about it and decided that it would make 
a useful teacher evaluation tool. They inserted themselves into the process 
in the role of GIFT-giver. This misuse of the GIFT converts evaluation from 
non-judgmental, formative assessment of learning based on trust into a judg¬ 
mental presumption about teaching based on doubt. Administrative misap¬ 
plication of the GIFT process happened more than once during the project 
period, and both the fact and surrounding rumors curtailed use of this meth¬ 
od. From an evaluator’s perspective, formal administrative support for the 
project must come with an explicit understanding that teachers must be al¬ 
lowed experiment with instructional methods free from premature summa- 
tive evaluation and the personal and professional risks it entails. 

Additionally, preliminary findings from the GIFT/QCD raise ques¬ 
tions about what helps and hinders learning in the science classroom. The 
idea that the best way to convey science content is through animated lectures 
packed with facts may not align with how middle, junior high, and high 
school students learn best. Students reported slower-paced, active learning 
as more effective. Students also reported that rapid talking and rapid pre¬ 
sentation hindered grasp of new material. Their reports also suggested that 
quiet, in-class reflection time was an important contributor to deeper learn¬ 
ing. Teachers may well produce better student achievement simply by slow¬ 
ing down and resisting pressures to “cover” more content superficially, pro¬ 
viding gaps of time for students to reflect and consolidate their knowledge, 
and managing sensory stimuli in the classroom and laboratory. Some of the 
GIFT/QCD findings are easily implemented; others are not. But in all cases, 
evaluation of factors that affected the classroom were raised to conscious¬ 
ness and made available for reflection and renewal. 

A Final Thought on Assessment 

Many students who attend SIUE come to campus from high 
schools served by the activities of this project. Some of those students 
have been taught by project teacher-participants. This presents us with an 
opportunity to explore the college readiness that the network generates. 
Although doing so is not straightforward, the principal investigators have 
followed students in first- and second-semester general chemistry courses 
that are engaged in the project through the involvement of their teachers. 
The results of this tracking assessment are shown in Table 2. This is ex¬ 
ploratory data for us to consider how the university partner is affected by 
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the network. These performance percentiles are not ideal as measures of 
learning, but the table is suggestive and encourages us to find ways to ob¬ 
serve how a regional network of science teachers may support science ed¬ 
ucation for both school and university partners. 

Table 2 


SUIE First and Second Semester Chemistry Performance Percentiles by 
Student Origin 


Origin of students 

Number of 
Students 

Average final chemistry 
perfonnance percentiles 

Took high school science 

359 

71.94% 

Took high school science in school 
served by Hands-On project 

97 

74.85% 

Took high school science in school and 
from teacher served by Hands-On project 

29 

80.75% 


Taken together, the multiple assessments of this grant-supported 
project provide some evidence that the Hands-On project produces improve¬ 
ments in student learning as students from project teachers appear to do bet¬ 
ter in first year chemistry courses. We do not understand the mechanisms 
for this, but we intend to follow-up with first year students from the three 
groups. This project is the only current ITQ project that has been in place 
long enough to begin to evaluate long-term effects, and because it is a high 
school project in part, we can follow-up and see how the project benefits both 
partners. We believe that long-term, realistic improvements in student sci¬ 
ence learning are achievable and that with a regional network system where 
partnerships exist, the university and schools can be of mutual benefit. 

Conclusion 

Of the ITQ projects, Hands-On Science for middle, junior high, 
and high school teachers at Southern Illinois University Edwardsville is 
the longest-lived, beginning more than 25 years ago as an Eisenhower 
grant. The Eisenhower grants were one late 20th century policy aimed 
at improving science education. In 2011, the emphasis on improving 
STEM education in the United States grows and shows no sign of subsid¬ 
ing. There is a general recognition that STEM matters in the hot, flat, and 
crowded 21st century (Darling-Hammond, 2010). The SIUE project has 
changed leadership several times, from Emil Jason to Sadegh Khazaeli, 
and now including Eric Voss, demonstrating the SIUE commitment as a 
partner and contributing to the stability that has made professional learn¬ 
ing among science teachers in regional networks possible. University pro¬ 
fessors worked directly with school teachers and developed a network of 
science teachers in settings as diverse as suburban St. Louis on both sides 
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of the Mississippi, urban East St. Louis, and dozens of rural communities 
hard hit by economic and demographic trends. Through IB HE support and 
encouragement, the project now uses evaluation for project planning, im¬ 
plementation, and improvement. 

The project began as a single-tier structural configuration with 
simple professor-to-teacher implementation through content-focused sum¬ 
mer institutes. No deans, superintendents, principals, or other constituen¬ 
cies were involved except as administrative functionaries. Early on, there 
were no particular requirements or supports for evaluation from IBHE. 
The professional development picture now emerges as more complex than 
just content expertise offered by professors to teachers. We can look back 
over the quarter century of the project, and consider how the five empiri¬ 
cally-grounded features of professional development have contributed and 
hindered our success: content focus, active learning, duration, collective 
participation, and coherence. 

We began with the first two of these: content focus and active 
learning, and continue to see their value. Sciences, technology, and en¬ 
gineering are dynamic fields always altered by new findings and direc¬ 
tions. The emphasis on test scores and prescribing what must be “cov¬ 
ered” work against the best practices for STEM teaching and professional 
development. Years of school improvement focused on literacy and math 
have left science out in the cold. Science is not emphasized in state test¬ 
ing, so it does not get the recognition its significance in the 21st century 
suggests it should receive. We also have attended to duration as a factor, 
offering summer workshops and year round study, always allowing teach¬ 
ers to continue, new teachers to join, and never turning anyone away. In 
the course of the last three years under IBHE’s new and emerging evalu¬ 
ation policies, we recognize the value of the regional networks that have 
developed and that provide necessary supports within schools or through 
ongoing contacts for those without science colleagues at work, recogniz¬ 
ing how we had encouraged collective participation. We have no coaches 
who work in the schools, as many ITQ projects do. The teachers are net¬ 
worked to support each other, and we have done this long enough that this 
seems to be working. We know that the potential can be more fully tapped. 
We continue to explore ways to make the network function and to increase 
its supportive features. 

Of the five features, the most challenging for Hands-On Science is 
coherence with policies that cause fragmentation at each level of the sys¬ 
tem, from the state legislature into the districts and schools, where leaders 
may or may not recognize or understand how the project can support their 
educational renewal and school improvement goals. As a result, we rec¬ 
ognize the need to be increasingly intentional about adopting a multi-tier 
configuration that brings regional leaders into the project (Baker, 2011). 
We have the reputation of the program in our favor, but challenges remain. 
The creation of the Program Board was a step in this direction. We gar- 
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nered internal support on the SIUE campus from both the Arts and Sci¬ 
ences and Education units, with representatives of the two deans serving 
on the board. We have science teachers who serve in formal and informal 
leadership roles serving on the board and working in the larger schools and 
districts to expand the program, which attracts more and more applicants 
each year. We have the trust of teachers and teacher leaders in the region. 
These are our most valuable assets. 

Implications for the future include development of teacher leader¬ 
ship and networking school and district leaders. In addition, we are begin¬ 
ning to consider the effects of the project on the university partner through 
new evaluation. The project began as a faculty-to-teacher configuration 
that added several sciences to the original chemistry program, improved 
the chemistry performance of students, and linked two colleges in a sci¬ 
ence education partnership. We also have preliminary connections to busi¬ 
ness partners with an interest in STEM and graduates from high school 
and college. Ideally, these graduates enter the workforce, and further edu¬ 
cation with a STEM foundation can serve them well. 

We are in a position to reap the rewards of years of partnership be¬ 
tween professors and teachers, recognizing that we are already reconfigur¬ 
ing the structure of our partnership and helping everyone from the school 
and university to evaluate results. We take this special issue of Planning 
and Changing as an opportunity to do some stocktaking and acknowl¬ 
edge what we have accomplished by working as partners with the science 
teachers in our region. 
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